Internet bots are software applications that perform automated tasks over the Internet. They are also known as web robots or bots and have been used in a variety of Internet applications. Web crawlers use bots to automatically fetch and index web pages from various web sites so that search engines can perform queries on the indexed web pages. Bots have been used in online auction sites to discover bargains. In online gaming sites, bots have been used where a response speed faster than that of human is desirable. In instant messenger applications, bots have been used to answer routine questions such as questions related to weather conditions, sporting event scores, and currency conversions.
Bots, however, may be used for malicious purposes. Bots have been used in automated operations to attack networked computers. For example, in a “denial of service” attack, a large number of bots, which may be geographically distributed, may saturate the target server with external communication requests so that the target server cannot respond to legitimate requests. Bots may also be used to influence ranking of search engine results. Search engines may use click data on search results to improve search results ranking. Bots may be used to repeatedly access certain search results from a large number of web sites to influence the ranking of the accessed search results.
Malicious bots may also be created in order to repeatedly access advertising-supported links to intentionally create the false appearance of many web site visits by human viewers. Advertising-supported links are generally short segments of text that are linked to an advertiser's web site. When a human viewer clicks on an advertising-supported link, the viewer “clicks through” the text link to visit the advertiser's web site. Many advertising-supported web sites have offered to sell advertising on a pay-per-click basis wherein the advertising-supported web site is only paid when a viewer “clicks through” on an advertising-supported link. When malicious bots are used to repeatedly click on advertising-supported links on a web site, web site advertisers are erroneously charged. Such attempts to create fictitious clicks on advertising-supported links are known as “click fraud”.
In these and other contexts, a key factor to combat malicious bots is the ability to detect whether a communication request to a web site comes from an Internet bot or a human viewer. However, since a large number of requests from bots may not be malicious, and the volume of communication requests to a web site may be extremely high, it is a challenging task to find a scalable method to detect bots.